System and method for compensating for motion relative to a barcode

ABSTRACT

A barcode decoding system and method for compensating for motion of between an image sensor and a barcode to improve decoding of the barcode. The barcode decoding system includes an imager for capturing an image of the barcode, a motion sensor for collecting acceleration data and a processor that is configured to determine a velocity of the image sensor during the exposure period based on the acceleration data and a periodic motion model. The determined velocity is used to adjust the edge detection algorithm used to detect the barcode features in order to decode the barcode. The orientation of the captured barcode can also be determined in order to determine the velocity in a direction perpendicular to the barcode features.

FIELD

The present disclosure relates generally to a system and method forreading barcodes, more specifically the disclosure relates to barcodereading devices using image sensors.

BACKGROUND

Barcode symbols provide a fast and accurate means of representinginformation about an object. Decoding or reading a barcode isaccomplished by translating the pattern of barcode features, such asbars and spaces in linear (1D) barcodes or blocks or other features in atwo-dimensional (2D) barcode, into the corresponding numbers orcharacters. Barcodes are widely used for encoding information andtracking purposes in retail, shipping and industrial settings.

Barcodes are typically read by a handheld barcode reading device thatincludes a camera or imager (these terms being used interchangeablyherein) for capturing an image of the barcode. Since the reading deviceis handheld, the quality of the captured image can suffer due to handmovement of the reading device relative to the barcode during the imagecapture period. For example, movement in the direction perpendicular tothe bars and spaces with a linear barcode can result in a blurring thatcauses misdetection of the edges between the bars and spaces in decodingthe barcode. For 2D barcodes, motion in any direction can induce imageblur that results in misdetection of the barcode feature. The imageblurring problem can be exasperated when using low cost image sensorsthat require longer exposure periods to capture images with suitablesignal to noise ratios or when attempting to capture an image of abarcode under less than ideal lighting condition which also requires arelatively longer exposure period.

Optical stabilization techniques are often used to reduce blurringassociated with the motion of a camera during exposure to compensate formovement of the imaging device. Optical stabilization varies the opticalpath to the image sensor of the camera in response to movement. Thistechnology can be implemented in the lens itself, or by moving thesensor as the final element in the optical path. The key element of alloptical stabilization systems is that they stabilize the image projectedon the sensor before the sensor converts the image into digitalinformation. Optical stabilization techniques typically requireadditional sensors, such as gyroscopes and accelerometers, and devicesto move lenses or sensors in the optical path. These additional sensorsand devices create a number of potential disadvantages in a handhelddevice, such as increasing the expense of the handheld device,increasing the number of moving parts in the handheld device, possiblyreducing the overall reliability, and increasing the power consumptionwithin the handheld device.

Another known technique used to minimize motion blur relies on using anaccelerometer to detect pauses in hand motion. This technique onlyallows the camera to attempt to capture an image during a detected pausein motion. This technique requires knowledge of the expected motionpattern of hand jitter in order to correctly detect the pauses. Thistechnique is used by some cell phone cameras, one example being theGlogger VS2 software available for Nokia phones running on Symbian OS.Published U.S. Patent Application 20050236488 to Kricorissian, andassigned to the assignee of the present invention, discusses anotherimplementation of a delay-based system. These approaches have proven notto be practicable for barcode readers because they often create anunacceptable delay when capturing the image of the barcode. Furthermore,these approaches may still result in blurring of the captured image inlower light environments or when using a low-cost image sensor where alonger exposure time is required since these approaches do not preventblurring or compensate for blur causing movement, but instead merelyattempt to avoid capturing the image of the barcode when movement isoccurring.

An alternative approach to reduce image blurring is to reduce exposuretimes. However, imagers used in barcode reading devices are typicallylower cost image sensors that require relatively longer exposure times.In the scanning environments where barcode reading devices are typicallyused, the light levels are often not sufficient to use reduced exposuretimes to eliminate image blurring. Providing supplementary lighting,such as a flash or other light source, can reduce the exposure time, butwould result in increased power consumption by the handheld device witha commensurate reduction in the useful operating time of the batterypowered handheld device.

SUMMARY

Accordingly, there is a need for a system and method for compensatingfor motion between an image sensor and barcode when capturing an imageof the barcode.

According to a first aspect, there is provided a method for compensatingfor motion between an image sensor and a barcode, the method comprisingthe steps of capturing an image of the barcode using the image sensor;determining a velocity between the image sensor and the barcode based onacceleration data provided by a motion sensor, the velocity determinedfor an exposure period of the image sensor to capture the image;selecting at least one parameter for edge detection processing of thebarcode based on the determined velocity to compensate for blurring ofthe image due to the determined velocity; and detecting edges offeatures of the barcode in the captured image using the at least oneparameter to decode the barcode. Preferably, the motion sensor and imagesensor are included in a handheld barcode scanner. Also preferably, thestep of determining the velocity comprises calculating a previousvelocity of the image sensor, and adjusting the previous velocity withthe acceleration data to determine the velocity. Also preferably, thedetermined velocity is calculated using a motion model, such as aperiodic or quasi-periodic motion model. Also preferably, the methodfurther includes the step of determining the velocity in a directionsubstantially perpendicular to edges of features of barcode.

According to a second aspect of the present invention, there is provideda barcode decoding system for capturing and decoding a barcode, thesystem comprising: an image sensor for capturing an image of thebarcode; a motion sensor for collecting acceleration data for motionbetween the image sensor and the barcode; and a processor configured todetermine a velocity of the motion between the image sensor and thebarcode based on the acceleration data, the velocity determined for anexposure period of the image sensor to capture of the image, theprocessor further configured to select at least one parameter for edgedetection processing of the barcode based on the determined velocity tocompensate for blurring of the image due to the determined velocity, andthe processor further configured to detect edges of features of thebarcode using the at least one parameter to decode the barcode.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will now be described, by way ofexample only, with reference to the attached drawings, wherein:

FIG. 1 is a perspective view of a handheld computing device;

FIG. 2 is a rear view of the handheld computing device of FIG. 1illustrating motion of the handheld computing device relative to abarcode;

FIG. 3 is a block diagram of a barcode decoding system illustrating theinterconnection of the functional subsystems;

FIG. 4 is a series of four graphs illustrating the effects of thevelocity between an image sensor of a barcode scanner and a barcode whencapturing an image of a linear barcode;

FIG. 5 is a block diagram of relevant elements of a barcode decodingsystem that provide motion compensation and barcode decodingfunctionality; and

FIG. 6 is a flowchart illustrating a method for compensating for motionbetween an image sensor of a barcode scanner relative and a barcode toimprove barcode decoding.

DESCRIPTION OF VARIOUS EMBODIMENTS

It will be appreciated that for simplicity and clarity of illustration,where considered appropriate, numerous specific details are set forth inorder to provide a thorough understanding of the embodiments describedherein. However, it will be understood by those of ordinary skill in theart that the embodiments described herein may be practiced without thesespecific details. In other instances, well-known methods, procedures andcomponents have not been described in detail so as not to obscure theembodiments described herein. Furthermore, this description is not to beconsidered as limiting the scope of the embodiments described herein inany way, but rather as merely describing the implementations of variousembodiments described herein.

The embodiments of the systems, devices and methods described herein maybe implemented in hardware or software, or a combination of both. Someof the embodiments described herein may be implemented in computerprograms executing on programmable computers, each computer comprisingat least one processor, a computer memory (including volatile andnon-volatile memory), at least one input device, and at least one outputdevice. For example, and without limitation, the programmable computerscan include handheld computing devices, or the combination of a portableor fixed computing device with a handheld barcode scanner where eitherthe computing device or the handheld barcode scanner can beprogrammable. Program code may operate on input data to perform thefunctions described herein and generate output data.

Reference is first made to FIGS. 1 and 2 which show a handheld computingdevice 100 having a barcode scanner 102 that provides optical barcodescanning functionality. Handheld computing device 100 can be any of awide range of digital devices that provides barcode readingfunctionality including, without limitation, devices which generatedigital information, such as computer terminals, RFID readers andoptical scanning devices, including dedicated barcode scanning devices,digital photo and document scanners. Handheld computing device 100 is ahandheld portable device, such as a mobile computer, mobile phone,handheld terminal, digital camera, scanner or other electronic deviceconfigured to capture and decode barcode images. Barcode scanner 102, asdescribed in further detail below, may comprise a set of hardware,firmware, and system software, employed in any suitable combination inorder to capture an image of barcode 120 using an image sensor.

Handheld computing device 100 can further include a keyboard 104 foruser input, a display screen 106, and an expansion port 108. Examples ofexpansion port 108 can include a Universal Serial Bus (USB) port orother similar expansion port for coupling compatible peripheral devicessuch as, but not limited to, a communication and synchronization cradlefor handheld computing device 100.

As used herein, the term barcode refers to an optical machine-readablerepresentation of information. Typically, barcodes encode information inthe widths and the spacing of parallel lines, sometimes referred to asbars, and may be referred to as linear or 1D (one-dimensional) barcodesor symbologies. Barcode 120 is provided as an example of a linearbarcode. Barcodes can also encode information in patterns of squares,dots, hexagons and other geometric shapes or symbols within imagestermed 2D (two-dimensional) matrix codes or symbologies. Although 2Dbarcodes use features other than bars, they are generally referred to asbarcodes as well. Accordingly, the barcode images discussed herein foruse with barcode scanner 102 can refer to either 1D or 2D barcodes. Aswill be described, the barcodes can be read by an optical scanner orreader referred to collectively as barcode scanner 102. As used herein,the objects used to encode the information, such as bars, squares, etc.,are referred to as features. For example, the features of barcode 120can refer to the either the black or white bars that comprise barcode120. Barcodes can also be black-and-white or color.

Arrow 122 is used to represent motion of barcode scanner 102 andhandheld computing device 100 relative to barcode 120. It is difficultfor a user to maintain handheld computing device 100 steady whilecapturing barcode 120 and this movement can induce motion blur in thecaptured image that could affect the ability of handheld computingdevice 100 to decode barcode 120.

Referring now to FIG. 3, a block diagram of a barcode decoding system300 is shown illustrating the interconnection of the functionalsubsystems. Handheld computing device 100 can include barcode decodingsystem 300 to provide barcode decoding and other functions. Barcodedecoding system 300 comprises a processor 302 that controls generaloperation of the system. Processor 302 interacts with functional devicesubsystems, which can include subsystems such as display screen 106,flash memory 304, random access memory (RAM) 306, auxiliary input/output(I/O) subsystems 308, serial port 310, keyboard 104, speaker 312,short-range communications subsystem 314, such as Bluetooth™ forexample, and expansion port 108. Barcode decoding system 300 can includea power source such as battery module (not shown) that can be removableand replaceable. While the illustrated embodiment of barcode decodingsystem 300 includes the functional subsystems described above, it willbe apparent to those of skill in the art that barcode decoding system300 can omit some of these subsystems and/or can include additionalsubsystems as required to meet an intended field of use for barcodedecoding system 300.

Barcode decoding system 300 can have the capability of communicating atleast data, and possibly any of data, audio and voice communications, toand from other devices connected by a communication network, as well asdata acquisition sources within a communication network. Barcodedecoding system 300 can include wired or wireless communicationcapability. In the wireless configuration, barcode decoding system 300typically includes radio frequency (RF) communication subsystem 316,which includes a receiver, a transmitter, and associated components,such as one or more embedded or internal antennae, and a processingmodule such as a digital signal processor (DSP) or the like. As will beapparent to those skilled in field of communications, the particulardesign of RF communication subsystem 316 depends on the specificcommunication networks in which barcode decoding system 300 is intendedto operate, and can include communication functionalities such asradio-frequency identification (RFID), Wi-Fi WLAN based on IEEE 802.11standards, Zigbee, Z-Wave, GSM EDGE, 1EVDO, HSPDA, and the like.

Still with regard to FIG. 3, operating system software can be executedby processor 302 that is stored in a persistent storage such as flashmemory 304, or alternatively, in other read-only memory (ROM) or similarstorage elements (not shown). Those skilled in the art will appreciatethat an operating system, specific device applications, or partsthereof, may be temporarily loaded into a volatile store such as RAM306.

Processor 302, in addition to its operating system functions, can alsoenable execution of software applications on barcode decoding system300. A predetermined set of applications, which control basic deviceoperations, or more customized, advanced device operations, may beinstalled on barcode decoding system 300 during its manufacture, such asduring the components configuration process.

Display screen 106 of barcode decoding system 300 may be used tovisually present a software application's graphical user interface (GUI)to a user via display screen 106. Display screen 106 can employ a touchscreen display, in which case the user can manipulate application databy modifying information on the GUI using direct touches by a finger orstylus. Depending on the type of handheld computing device 100, the usermay have access to other types of input devices, such as, for example, ascroll wheel, trackball or light pen.

A graphical user interface presented on display screen 106 of barcodedecoding system 300 may enable an operator or administrator to interactwith handheld computing device 100. It is further contemplated thatbarcode decoding system 300 may be communicatively coupled to a remotelylocated database (not shown).

Barcode decoding system 300 further comprises barcode scanner 102.Barcode scanner 102 can be integrated with handheld computing device 100as shown in FIGS. 1 and 2. In other embodiments, barcode scanner 102 cancomprise a secondary handheld enclosure that can includes motion sensor318 and some of other the subsystems illustrated in FIG. 3. For example,barcode scanner 102 can be a pistol scanner that is communicativelytethered to barcode decoding system 300. The pistol scanner can includemotion sensor 318 and image sensor 320, and the pistol scanner canfurther include a trigger and logic to control the operation of thepistol scanner. The pistol scanner can be tethered to barcode decodingsystem 300 either by wire, such as a USB link, or through a wirelessprotocol that communicates with short-range communication subsystem 314of handheld computing device 100. The term “handheld barcode scanner” asused herein can refer to either barcode scanner 102 integrated withinhandheld computing device 100 or a secondary handheld enclosure withbarcode scanner 102 that can connect to the subsystems of barcodedecoding system 300.

Barcode scanner 102 can comprise any suitable combination of software,firmware and hardware to implement scanning of barcode 120 using imagesensor 320. Image sensor 320 can further include a light source and alens (not shown). The light source and lens can be controlled by imagesensor 320 or processor 302. The lens can be a fixed focus lens with asuitable depth of field, or can be an auto-focus lens. Image sensor 320can be a CCD, CMOS or other suitable image sensor.

If barcode scanner 102 is implemented in a secondary handheld enclosure,barcode scanner 102 can further include a motion sensor 318 that canprovide acceleration data with respect to the motion of barcode scanner102. Acceleration data is used by barcode scanner 102 to recover thevelocity of barcode scanner 102. If barcode scanner 102 is integratedwith handheld computing device 100, motion sensor 318 can be a separatesubsystem connecting to barcode decoding system 300 in order to provideacceleration data with respect to the motion of barcode scanner 102. Insome embodiments, motion sensor 318 can be an accelerometer thatprovides linear/translational acceleration data in one or more planes ofmovement according to the axes of the accelerometer (e.g. x, y, and z).In other embodiments, motion sensor 318 can be a gyroscope that has axesto measures pitch, yaw and roll and provides acceleration data in theform of degrees per second. Using the distance from barcode 120, whichcan be estimated based on focal length or other factors, angularacceleration data from the gyroscope can be used to approximatetranslational acceleration data similar to that obtained from anaccelerometer. A gyroscope can also provide positional orientationinformation relative to a reference position. Acceleration data may beprovided from both an accelerometer and a gyroscope to provide moreaccurate acceleration data. Motion sensor 318 can be implemented as aMEMS sensor and can include either an accelerometer, a gyroscope, or acombination thereof in a single part. Velocity can be determined fromthe acceleration data provided by motion sensor 318 as will be describedfurther below.

Acceleration data can further include timestamp data that may be used todetermine velocity. Depending on where acceleration data is processed,motion sensor 318 can provide acceleration data to processor 302 ordirectly to barcode scanner 102.

In most cases it is assumed that barcode 120 is static (or moving aninsignificant amount) and the motion of barcode scanner 102 and imagesensor 320 with respect to barcode 120 causes image blur in the captureimage. However, in some embodiments, barcode 120 may be moving withrespect to a static barcode scanner 102. In that case, motion sensor 318can be used to provide movement information of barcode 120 to capturemotion of barcode 120 as opposed to motion of barcode scanner 102 andimage sensor 320. For example, if image sensor 320 is statically mountedto capture images of barcode 120 placed on a moving object, such as aconveyor or platform, motion sensor 318 can capture this movement andprovide it to barcode decoding system 300. For example, motion sensor318 can provide acceleration data of a conveyor or platform that barcodedecoding system 300 can use to determine the velocity between imagesensor 320 and barcode 120 (i.e. the velocity of the conveyor orplatform in this case).

Referring now to FIG. 4, a series of four graphs are shown thatillustrate the effects of the velocity between image sensor 320 andbarcode 120 when capturing an image of a linear barcode. Each graphillustrates reading the start/stop pattern of a standard C39 barcode ata different velocity, v, with respect to image sensor 320. The verticalaxes represents increasing level of brightness/lightness (i.e. upperlevel representing white and lower levels representing black).

The C39 barcode standard uses a fixed ratio between the widest andthinnest black and white bars that barcode decoding system 300interprets to decode the codeword. The C39 start/stop patternillustrated in FIG. 4 is made up of the following sequence of bars:narrow black; wide white; narrow black; narrow white; wide black; narrowwhite; wide black; narrow white; and narrow black. Conventionally, howaccurate this array of widths is recreated by image sensor 320 ofbarcode scanner 102 affects the accuracy of interpreting the codewordand the entire barcode 102. Motion between image sensor 320 and barcode120 can result in motion blur in the captured image that causes errorswhen decoding barcode 120. The motion blur is proportional to thevelocity of the motion. Other barcode standards can encode informationusing other known methods and are similarly prone to errors from motionas described below.

First graph 400 illustrates the case where the velocity of barcode 120with respect to image sensor 320 is zero, i.e. v=0. Detection of blackand white bars of barcode 120 can be performed fairly accurately usingan edge detection algorithm if the image of barcode 120 has a highsignal to noise ratio so that there is sufficient contrast. The verticallines in the signal indicate a strong contrast between the black andwhite bars.

Graphs 401, 402, and 403 illustrate effects of increasing velocity (i.e.v=1; v=2; and v=3, where higher values for v indicate greatervelocities) between barcode 120 and image sensor 320. The velocity ineach graph is related to the slope of each edge of the black and whitebars of barcode 120 such that a larger velocity or slope represents ablurrier edge between white and black bars in the image. With a fairlylow speed, an edge detection algorithm can still perform relatively wellas the signal to noise ratio is still relatively good since a movingtarget can help average and cancel noise from sources such as dirt inthe optical path, light distortion, print quality, cross-talk, badpixels, etc. Motion causes these noise sources to be distributed andaveraged over multiple pixels of image sensor 320. An edge detectionalgorithm can be adjusted for a lower or higher velocity as will bedescribed below. An edge detection algorithm adjusted for a lowervelocity may fail at a higher velocity due to a lower signal amplitudewhile an edge detection algorithm adjusted to work at a higher velocitymay fail at a lower velocity since it is more susceptible noise. Failureto properly adjust the edge detection algorithm for the appropriatevelocity will result in failure to detect edges of the black and whitebars and failure to decode barcode 120. Alternatively, a different orvariation of the edge detection algorithm can be selected based onvelocity.

Graphs 400, 401, 402, and 403 illustrate how the upper and lowerdetection limits of an edge detection algorithm, shown as horizontallines, can be adjusted to accurately detect the edges of features ofbarcode 120. Selecting a higher detection threshold, such as that shownin graph 400 where the upper detection limit for detecting a black towhite transition is near the maximum signal level and the lowerdetection limit for detecting a white to black transition is near theminimum signal level, helps to filter noise from the captured image andperforms well in high contrast and/or low velocity conditions. Graphs401, 402, and 403 illustrate selecting a lower detection threshold wherethe upper detection limit and lower detection limit are adjusted toaccurately capture the width of the bars from the signal. The upperdetection limit captures a white bar and the lower detection limitcaptures a black bar. Although the lower detection threshold has lessdistance between the upper and lower detection limits that may make thedetection algorithm more susceptible to noise, a lower detectionthreshold can be used for higher relative velocity conditions wherenoise in the optical path is averaged over multiple pixels of imagesensor 320.

Edge detection processing based on thresholds is provided as an exampleof how knowledge of the velocity between barcode 120 and image sensor320 can be used to adjust the edge detection processing. Other edgedetection algorithms that use a different edge detection mechanism canalso be adjust based on the velocity information. Some edge detectionalgorithms use a derivative of the scan to determine the edges of thefeatures of barcode 120 (i.e. where the second derivate is zeroindicates a change in contrast between the features of barcode 120).Since higher velocities can effectively cancel spurious noise, thesederivative-based edge detection algorithms can also use the velocitybetween image sensor 320 and barcode 120 to adjust the edge detectionprocessing such that the algorithm can be adjusted to be more sensitiveto changes in the derivatives to determine the edges of the features ofbarcode 120. Accordingly, other edge detection algorithms can beadjusted based on the velocity between the image sensor 320 and barcode102 to either compensate for the cancelled spurious noise in thecaptured image or the less prominent edges of features of barcode 120 inthe captured image.

Referring now to FIG. 5, a block diagram 500 of relevant elements ofbarcode decoding system 300 that provide motion compensation and barcodedecoding functionality is shown. Image sensor 320 and motion sensor 318can be implemented as separately packaged parts, provided only that,motion sensor 318 must move with image sensor 320. Remaining elements,such as barcode orientation extractor 502 velocity vector calculator504, and edge detection decoder 506, can be implemented either togetheror separately, and, at least in part, as software code executed onprocessor 302 or implemented in custom logic, such as an ASIC orprogrammable logic device, including but not limited to FPGAs or DSPs.The term “processor” as used herein can refer to a microprocessor, suchas processor 302, or a custom logic solution, such as an ASIC or FPGA.

Image sensor 320 captures an image of barcode 120. Typically, thecapture operation is initiated through a user's interaction withhandheld computing device 100 or barcode scanner 102. The captured imagetypically includes barcode 120 and the surrounding background.

The captured image from image sensor 320 is then provided to barcodeorientation extractor 502 that determines the orientation of barcode 120in the captured image. Barcode orientation extractor 502 processes thecaptured image to determine the orientation of barcode 120 within thecaptured image based on, for example, the skew of barcode 120 or theangle of features of barcode 120. The determined orientation of thecaptured image of barcode 120 is provided to velocity vector calculator504.

Velocity vector calculator 504 uses acceleration data provided frommotion sensor 318 to determine velocity, as will be described below.Velocity vector calculator 504 uses the determined orientation and thedetermined velocity, to determine the velocity of barcode scanner 102 ina direction perpendicular to barcode features (i.e. the long side of barin a 1D barcode or the side of a cell in a 2D barcode).

Barcode orientation extractor 502 can determine the orientation of barsof a 1D barcode or the squares or rectangles of a 2D barcode using anysuitable image processing algorithm as will occur to those skilled inthe art. In one embodiment, image processing can convert the capturedimage to binary using dynamic thresholding and then use chain-codeprocessing to effectively outline the features of barcode 120 in thecaptured image. The result of chain-code processing for each feature ofbarcode 120 will be the smallest wrapping rectangle. The orientation ofthese rectangles can then provide the best statistical orientation ofthe features of barcode 120.

Alternatively, barcode orientation extractor 502 can use imageprocessing based on the correlation of multiple scan directions. First,the image processing algorithm can detect the location of barcode 120within the captured image. Next, several short scans are performed atdifferent angles through the detected location to detect the edges ofthe features of barcode 120. Scans at angles that have the highestmutual correlation correspond to the angle that is perpendicular to thefeatures of barcode 120 to determine orientation.

In other embodiments, barcode orientation extractor 502 can also beprovided with position data from motion sensor 318 to determine theposition of barcode scanner 102 in free space (relative to a referenceposition) to assist with determining the orientation of barcode 120based on a presumed or statistical orientation of barcode 102. Forexample, if the majority of barcodes are parallel with the horizon, thisinformation can assist the orientation extraction process. Barcodeorientation extractor 502 can also provide a preprocessed image to edgedetection decoder 506 such that the image has been de-skewed and/orbackground information removed.

Velocity vector calculator 504 determines the velocity of image sensor320 that is associated with motion sensor 318. The velocity ispreferably calculated for the exposure period of image sensor 320 whencapturing the image of barcode 120. The exposure period includes thetime that image sensor 320 is collecting light to capture the image. Asused herein, the term “exposure period” can also include the timeimmediately prior to or after capturing the image such that the velocitycan be calculated using either prior or post measurements, an average orany other combination thereof.

Determining the velocity of image sensor 320 uses data from motionsensor 318 and then determines the velocity with respect to a directionperpendicular to the edges of features of barcode 120. Velocity can becalculated by applying digital integration to the translationalacceleration data from motion sensor 318. Velocity vector calculator canalso use the distance between the image sensor 320 and barcode 120 incalculating velocity if motion sensor 318 includes a gyroscope thatprovides angular acceleration (i.e. pitch, yaw or roll in degrees persecond). The distance can be derived from the variable or managed focusinformation from image sensor 320.

If barcode 120 is a linear barcode, velocity vector calculator 504determines the velocity of image sensor 320 in a direction perpendicularto the edges of the bars that comprise barcode 120 in the capturedimage. With a 2D barcode, velocity vector calculator can determine avelocity in directions perpendicular to the edges of the 2D barcodefeatures. Most current 2D barcode have orthogonal features (i.e. squaresor rectangles) but other variations can use triangles, hexagons or othergeometric shapes that can require determining the velocity in more thantwo directions.

Velocity vector calculator 504 uses acceleration data provided by motionsensor 318 to adjust a previously calculated velocity of image sensor320. Velocity can be continuously updated by acceleration data frommotion sensor 318. Velocity vector calculator 504 can use a motionmodel, such as a periodic (or quasi-periodic) motion model thatsimulates motion of the human hand, to assist with determining thevelocity from the acceleration data. Generally, velocity informationcannot be obtained strictly from acceleration data without knowledge ofan initial velocity. Using a motion model allows velocity vectorcalculator 504 to recover velocity from acceleration data. For example,in a simplified pendulum motion model when acceleration changes from onedirection to an opposite direction, the simplified pendulum motion modelprovides that the object had a velocity of zero some time between thechange in direction of acceleration. A motion model can be selectedbased on acceleration data or position information from motion sensor318 so that velocity can be determined accurately based on accelerationdata. For example, position data based on acceleration data from motionsensor 318 can also be used to select an appropriate motion model basedon the orientation of how barcode scanner 102 is being held by anoperator.

Periodic motion models are based on the fact that most motion activitiesin the human body are happening at 8 Hertz and below. Periodic motionappears in hand jitter patterns that are a common source of image blurwith a handheld image sensor 320. Typically, image sensor 320 capturesan image in 10-20 ms in average light condition (e.g. an officeenvironment with a brightness of approximately 300 lux) this provides acapture time that is less that ⅛ of the period of human body motion.Using a periodic motion model, velocity vector calculator 504 caninterpolate a constant velocity (assuming no acceleration between imageparts) during the period that the image was captured by image sensor320.

One example of a motion model can include a pendulum-like motion whereacceleration and velocity are shifted relative to each other by aquarter period and their amplitudes are proportional to each other.Other embodiments can include other motion models or more complicatedmotion models that better fit the acceleration data received by velocityvector calculator 504. For example, in some embodiments velocity vectorcalculator 504 can select a motion model or frequency based on receivedacceleration data from motion sensor 318. Preferably, at least twoperiods of periodic motion are used to select a frequency. The receivedacceleration data can include acceleration data for past image capturesand can be associated with a user profile so that a motion model may berefined for a particular user.

Edge detection decoder 506 processes the captured image using an edgedetection algorithm to determine the edges of the features of barcode120 in order to decode barcode 120. Edge detection decoder 506determines at least one parameter for edge detection processing of thebarcode 120 based on the velocity (including direction(s)) received fromvelocity vector calculator 504. Edge detection decoder 506 can make thisdetermination based on the velocity perpendicular to the features ofbarcode 120 provided by velocity vector calculator 504. Edge detectiondecoder 506 determines the amount of motion-induced image blur using thedetermined velocity received from velocity vector calculator 504 and anyone or combination of the following factors including, but not limitedto: size of features of barcode 120; the resolution of image sensor 320;and the focal distance of the lens. The amount of velocity induced bluris used to select parameters to adjust and/or select the edge detectionalgorithm.

For example, edge detection decoder 506 can use the resolution of imagesensor 320 and the determined velocity to determine that 2.5 pixels areaffected by gradient shading due to the determined velocity provided byvelocity vector calculator 504. If the minimum size of features ofbarcode 120 is 10 pixels, then no adjustment, or a very minimaladjustment, would be required to adjust for the motion-induced imageblur. However, if the minimum size of features of barcode 120 is 3pixels then edge detection decoder 506 may select parameters to adjustthe upper and lower detection limits of the edge detection algorithm.The appropriate parameter(s) is/are selected to adjust the edgedetection algorithm (or to select the appropriate edge detectionalgorithm) to accurately detect the edges of features of barcode 120 tocompensate for the motion-induced image blur from the determinedvelocity.

Edge detection algorithms typically operate to detect a change incontrast at a point in the image (e.g. between a black and white bar inbarcode 120) and then compares that change in contrast to a detectionlimit to determine whether an edge exists. Edge detection decoder 506can adjust these detection limits based on the velocity of barcodescanner 102.

As demonstrated in FIG. 4, higher velocities can require a lowerdetection threshold. Once the edges of the features of the barcode 120are detected, the barcode features can be identified in order to decodebarcode 120.

The detection thresholds as illustrated in FIG. 4 are provided as oneexample of a parameter for edge detection processing. Other parametersused to adjust edge detection processing can include edge position, edgeshape or edge pattern. For example, the edge shape can be affected bynoise from image sensor 320 or noise inherent in barcode 120 that may bedue to dirt in the optical path, light distortion, print quality, etc.that the algorithm needs to separate from the motion-induced image blur.Edge pattern information can be used to tune the algorithm based on astart/stop pattern or a fixed ratio between the bars and spaces. Anotherparameter could be used to select an appropriate edge detectionalgorithm such as, for example, using a differential edge detectionalgorithm.

Referring now to FIG. 6, a flowchart 600 is shown illustrating anexample method for compensating for motion between image sensor 320 ofbarcode scanner 102 and barcode 120 to improve barcode decoding. At step602, an image of barcode 120 is captured using image sensor 320 ofbarcode scanner 102. Capturing the image is typically initiated by auser of handheld computing device 100 through a user interfaceinteraction, such as pressing a key of keyboard 104 or specificallytasked button or trigger of barcode scanner 102.

At step 604, the velocity of barcode scanner 102 is determined duringthe period of time the image was captured by image sensor 320. Dependingon the construction of image sensor 320, ambient lighting conditions andthe type and printing of barcode 120, image sensor 320 can take from 10to 60 milliseconds to capture the image. The velocity is determined fromthe acceleration data provided by motion sensor 318. Acceleration datais used along with a motion model in order to determine the velocity.The velocity of image sensor 320 is continually adjusted between (orjust prior to) capture events based on the acceleration data and themotion model.

At step 606 the orientation of features of barcode 120 in the capturedimage is determined. The captured image can be processed to determinethe orientation by detecting the skew or angle of barcode 120 and itsfeatures. Orientation information is used to determine the orientationof features of the barcode in the captured image with respect to theaxes of measurement of motion sensor 318 in order to determine velocityin a direction along barcode 120. This is calculated in step 608 wherethe orientation and velocity are used to determine the velocity of imagesensor 320 in a direction substantially perpendicular to edges of thefeatures of barcode 120. With a 2D barcode, a plurality of velocitiescan be determined such that each velocity is substantially perpendicularto one of a plurality of edge of features of the barcode. Withorthogonal 2D barcodes this involves calculating two velocities that areperpendicular to each other but other 2D barcodes can use non-orthogonalshapes (e.g. triangles) as barcode features that can require determiningthree different velocities. This can involve mapping a velocity vectorcalculated in step 604 to the direction of the edges of the features ofbarcode 120.

Once the velocity/velocities is/are determined with respect to the edgesof the features of barcode 120 in the captured image, in step 610 atleast one parameter is selected for edge detection processing of barcode120 based on the determined velocity to compensate for blurring of thecaptured image due motion between image sensor 320 of barcode scanner102 and barcode 120. The parameter can include a sensitivity thresholdthat is selected based on the determined velocity so that a lowersensitivity is selected for higher velocities. Other parameters of edgedetection processing that can be adjusted based on velocity can includeany one of edge position, edge shape, or an edge pattern.

At step 612 the edge detection processing commences using the providedparameters to process the captured image of barcode 120 to detect theedges of the features of barcode 120 in order to identify barcodesymbols for decoding.

While the embodiments have been described herein, it is to be understoodthat the invention is not limited to the disclosed embodiments. Theinvention is intended to cover various modifications and equivalentarrangements included within the spirit and scope of the appendedclaims, and scope of the claims is to be accorded an interpretation thatencompasses all such modifications and equivalent structures andfunctions.

1. A method for compensating for motion between an image sensor and abarcode, the method comprising the steps of: capturing an image of thebarcode using the image sensor; determining a velocity between the imagesensor and the barcode based on acceleration data provided by a motionsensor, the velocity determined for an exposure period of the imagesensor to capture the image; selecting at least one parameter for edgedetection processing of the image of the barcode based on the determinedvelocity to compensate for blurring of the image due to the determinedvelocity; and detecting edges of features of the barcode in the capturedimage using the at least one parameter to decode the barcode.
 2. Themethod of claim 1, wherein the image sensor and motion sensor areincluded in a handheld barcode scanner.
 3. The method of claim 2,wherein determining the velocity further comprises: calculating aprevious velocity of the image sensor; and adjusting the previousvelocity based on the acceleration data to determine the determinedvelocity.
 4. The method of claim 3, wherein calculating the previousvelocity and adjusting the previous velocity are based on a motionmodel.
 5. The method of claim 4, wherein the motion model is any one ofa periodic motion model and a quasi-periodic motion model.
 6. The methodof claim 3 further comprising: determining orientation of features ofthe barcode in the image with respect to axes of the motion sensor, andwherein determining the velocity is based on the orientation of featuresof the barcode and the axes of the motion sensor to determine thevelocity in a direction substantially perpendicular to edges of featuresof barcode.
 7. The method of claim 6, wherein the barcode is a 2Dbarcode having features with edges in a secondary direction, the methodfurther comprising: determining a second velocity based on theorientation of features of the barcode and the acceleration data fromthe two or more axes of the motion sensor to determine the secondvelocity substantially perpendicular to edges in the secondarydirection.
 8. The method of claim 2 wherein the at least one parameteris a detection threshold that is selected based on the velocity, whereina lower detection threshold is selected for a higher velocity.
 9. Themethod of claim 8 wherein the at least one parameter is any one of anedge detection algorithm, an edge position, an edge shape, or an edgepattern.
 10. A barcode decoding system for capturing and decoding abarcode, the system comprising: an image sensor for capturing an imageof the barcode; a motion sensor for collecting acceleration data formotion between the image sensor and the barcode; and a processorconfigured to determine a velocity of the motion between the imagesensor and the barcode based on the acceleration data, the velocitydetermined for an exposure period of the image sensor to capture theimage, the processor further configured to select at least one parameterfor edge detection processing of the image of the barcode based on thedetermined velocity to compensate for blurring of the image due to thedetermined velocity, and the processor further configured to detectedges of features of the barcode in the image using the at least oneparameter to decode the barcode.
 11. The system of claim 10, furthercomprising a handheld barcode scanner housing the image sensor andmotion sensor.
 12. The system of claim 11, wherein the processor isfurther configured to calculate a previous velocity of the handheldcomputing device; and adjust the previous velocity with the accelerationdata to determine the determined velocity.
 13. The system device ofclaim 12, wherein calculating the previous velocity and adjusting theprevious velocity are based on a motion model.
 14. The system device ofclaim 13, wherein the motion model is any one of a periodic motion modeland a quasi-periodic motion model.
 15. The system computing device ofclaim 12, wherein the processor is further configured to determineorientation of features of the barcode in the image with respect to theaxes of the motion sensor, and determine the velocity based on theorientation of features of the barcode and the axes of the motion sensorto determine the velocity in a direction substantially perpendicular toedges of features of barcode.
 16. The system computing device of claim15, wherein the barcode is a 2D barcode having features with edges in asecondary direction, the processor is further configured to determine asecond velocity based on the orientation of features of the barcode andacceleration data from the two or more axes of the motion sensor todetermine the second velocity substantially perpendicular to edges inthe secondary direction.
 17. The system computing device of claim 11,wherein the at least one parameter is a detection threshold that isselected based on the velocity, wherein a lower detection threshold isselected for a higher velocity.
 18. The system computing device of claim11, wherein the at least one parameter is any one of an edge detectionalgorithm, an edge position, an edge shape, or an edge pattern.
 19. Thesystem computing device of claim 11, wherein the motion sensor is anyone of an accelerometer and a gyroscope.